@) PUBLISHING 


v 


Indexed By : 


» 2D GID 


@ P-ISSN - 2830-6392 (PRINT) 
@ E-ISSN - 2962-2522 (ONLINE) 


JOURNAL OF GOVERNANCE, TAXATION 


A AND AUDITING 


ANALYSIS OF THE EFFECTIVENESS OF IMPLEMENTATION SAP 


R/3 IN PT. ABC 
Volume: 1 Anneke WANGKAR! 
Number: 2 TFaculty of Economics and Business Sam Ratulangi University, Indonesia 
Page: 89 - 96 Corresponding author: Anneke Wangkar 
E-mail: annekewangkar@unsrat.ac.id 
Article History: Abstract: 


Received: 2022-09-17 This research analyses the effectiveness of SAP R/3 toward user satisfaction in 

Revised: 2022-10-04 using the application. The research is in PT ABC, manufactured in paper and 

Accepted: 2022-11-18 tissue products. The research questions of this research are: "Is there any gap 
between the user expectation about the effectiveness of the application and the 
application's performance? Are some factors, which are integrated data, reliable 
report, processing time, and flow of information between departments 
significantly, affecting the user satisfaction in using SAP R/3?" The data for this 
research is collected using questionnaires to the employees of four departments 
in PT. ABC, which are the Sales and Marketing Department, Finance and 
Accounting Department, Production Department, and Logistics Department. The 
analysis used in the research is Gap Analysis and Multiple Regression Analysis. 
The results of this research are: "There is a gap between the user expectation about 
the effectiveness of the application and the application performance. Three factors 
significantly affect user satisfaction in using SAP R/3: integrated data, reliable 
report, and processing time." The benefit of the research is the effectiveness of the 
SAP R/3 implemented in PT. ABC. The research result can be used as a reference 
for further improvement of SAP R/3.The research results can be used as a 
reference for other companies or organizations implementing SAP R/3. 
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Satisfaction, User Satisfaction. 
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INTRODUCTION 


PT. ABC is a company that produces paper and tissue products. Information is the most 
crucial aspect that contributes to most business processes. In running the business process, PT. 
ABC had some problems because all business process activities were done manually. These manual 
activities caused problems such as unintegrated data caused by human error and different data in 
each department. The report has no standard format because each department uses its program 
application. So, doing their business process will take a lot of time. In order to overcome these 
problems and support the business process, PT. ABC decided to implement SAP R/3. 

ERP and SAP. Enterprise Resource Planner (ERP) is a business application designed to 
provide an integrated, systematic environment for a business to manage its daily activities. "ERP 
Systems link financial, manufacturing, human resources, distribution, and order management 
systems into a tightly integrated single system with shared data and visibility across the business" 
(Escalle & Cotteleer, 2022). 

SAP ('"Systeme, Awendungen, und produkte in Datenverarbeitung" or "Systems, 
Applications, and Products in Data Processing") was founded by five former IBM employees in 
1972 in Germany." (Escalle and Cotteleer, 2022). SAP R/3 (SAP release R/3), available since 1992, 
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is SAP's application that operates in a client/server architecture and uses a central relational 
database. 


Integration is an essential factor in the most system. "The meant of integration in SAP is that 
business groups can work with data in a connected system in real-time" (Prince, 2018). Using 
relational databases and integration in SAP can make SAP valuable, robust and famous in 
industries. SAP provides a flexible method to establish a business structure because it is designed 
to reflect any business organization. 

According to Phil Robinson (http://www.vbip.com), there are some characteristics of SAP 
R/3, which are: adaptability and scalable, data integrity, Electronic Data Interchange (EDI), self- 
sufficient, real-time, multi-user, client-server system, and modular. There are three functional areas 
in SAP R/3: financial, human resources, and logistics (Hernandez, 2015). 

IS Effectiveness Measurement. The Information System's (IS) effectiveness can be measured 
through IS usage, user information satisfaction (UIS), quality of decision-making, productivity 
from a cost/benefit analysis, and system quality (Grover et al., 2016). The IS success model by 
DeLone and McLean. It consists of six interdependent constructs: System Quality, Information 
Quality, Use, User Satisfaction, Individual Impact, and Organizational Impact (DeLone & McLean, 
2022). 

User Information Satisfaction. User satisfaction generally results from comparing user 
expectations of the IS with perceived performance. The gap is the discrepancies between 
expectations and performance. The perceived performance exceeds the expectation if the gap's 
value is positive. On the other hand, if the gap's value is negative, the perceived performance is 
below the expectation. In measuring UIS, some models exist, such as the Miller-Doyle approach, 
the Kim model, and a gap model (Remenyi et al., 2014). 

The Miller-Doyle Approach. According to Remenyi et al. (2014), the Miller-Doyle approach 
is designed to measure the perceived effectiveness of the overall IS function and involves the use 
of a questionnaire, which consists of five parts labeled Part A through Part E. "Part A consists of 34 
questions which measure the extent to which certain facets of the IS are perceived to be important 
in ensuring the Organization's IS will be effective and successful. The attitudes are rated on a 
semantic differential scale of 1 (irrelevant) to 7 (very critical). Part B consists of four questions on 
the future needs for IS; Part C consists of the same 34 questions as Part A, but in this case, the 
respondent is asked to rate the 34 questions concerning the actual performance achieved within 
their Organization. Again a 7-point scale is used, but in this case the levels of perceived importance 
go from 1 (very poor) to & (excellent); Part D consists of four questions relating to the 
Organization's performance in developing new systems; Part E consists of four questions which 
capture certain demographic data. There is also a question which asks for a raring of the 
Organization's overall performance on a scale of 1 (complete failure) to 7 (very successful)" 
(Remenyi et al., 2014). 

The Kim Model. In the Kim model, UIS is influenced by post-implementation experience 
and pre-implementation expectations of the IS (Remenyi et al., 2014). This model will be measured 
the UIS by the discrepancy between the user's perception score of the IS performance and the user's 
expectation score of the IS. This model will show how discrepancies during the developmental and 
service delivery processes influence the UIS. The developmental stage consists of two sub-stages: 
the determination of the IS requirements and the design and installation of the IS (Remenyi et al., 
2014). 

PT. ABC started to develop SAP R/3 at the end of 2020 and implemented it on June 2021. 
This research will analyze the effectiveness of implementing SAP R/3 in PT. ABC. The analysis 
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will be done on user satisfaction with using SAP R/3 to support their job. PT.ABC needs to know 
whether SAP R/3 gives a significant improvement in order to support the business process. The 
research questions are: 


1. Is there any gap between the user expectation about the effectiveness of the application and the 
application's performance? 


. Is the integrated data significantly affecting user satisfaction in using SAP R/3? 
. Is the reliable report significantly affecting user satisfaction in using SAP R/3? 
. Is the processing time significantly affecting user satisfaction in using SAP R/3? 


oF O N 


. Is the flow of information between departments significantly affecting user satisfaction in using 
SAP R/3? 


METHODS 


Population and Sample. The populations of the research are the SAP R/3 users at PT. ABC. 
Four departments will be chosen as the populations for the research: The sales and Marketing 
Department, Finance and Accounting Department, Production Department, and Logistic 
Department. These departments were chosen because the SAP R/3 has been implemented in those 
departments. The sample will be chosen randomly, not all SAP R/3 users in each department. The 
method that will be used is the multi-stage random sampling method. This method is chosen 
because the populations are only some of the company's departments. 

Time, Location, and Data Collection Method. The research began on August 2021 in PT. ABC, 
Manado. The data will be collected through a survey. The questionnaires were used to collect data 
for the research. The language used in the questionnaire is the Indonesian Language because the 
respondents are more familiar with Indonesia Language. 

Research Model. This research will use two research models: the model for the gap analysis 
(Figure 1) and the model for the regression analysis (Figure 3.2). The gap analysis determines user 
satisfaction by analyzing the gap between the user's expectations and application performance 
scores. 


User Application 
Expectation Performance 


: integrated data : integrated data 

: reliable report : reliable report 

: procesing time : procesing time 

: flow of information : flow of information 
between departments between departments 


Figure 1. Research Model for Gap Analysis 


The regression analysis is done to determine whether some factors, which are integrated data, 
reliable reports, processing time, and information flow between departments, affect user satisfaction 
in using SAP R/3. 
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Y= user satisfaction 


meee X4= flow of information 
X2= reliable report between departments 


Figure 2. Research Model for Regression Analysis 


Validity and Reliability. Data, which are obtained from the research, should be valid and 
reliable. Validity shows the degree of accuracy, which is the accuracy between actual data and 
collecting data by the researcher. Reliability shows the degrees of consistency, which is consistency 
in specific time intervals. 

Statistical Analysis. This research will use a Paired-sample T-test for the first research model 
and Multiple Regression Analysis for the second research model. 

Paired-Sample T-Test. The Paired-sample T-test is done for the difference between the two 
population means. The test for the difference between the two population means is (Aczel, 2008): 

Multiple Regression Analysis. Multiple regression analysis is a method of analyzing the 
change in one variable (dependent variable) by using a set of other known variables (independent 
variables) in order to estimate the mean value of the dependent variable based on the know values 
of the other independent variables (Galleries, 2016). The result of regression analysis is the R-squared 
(R2), the multiple coefficients of determination, which describes how well a set of variables explains 
a dependent variable. 

F-test (Anova test). F-test (Anova test) is used to test two samples or more. The assumption 
that used in F-test are (Santoso, S., 2017): 

1. The populations are normal distribution 
2. The Populations have same variance 
3. There is no relation between each sample 

The F-value and significance value results will be analyzed. If the significance value is equal 

to 0.000, it means that the regression as a whole is significant. 


RESULT AND DISCUSSION 

Survey Result. The questionnaires were sent to 130 respondents of PT. ABC, Manado. Of 130 
respondents, only 123 respondents sent back the questionnaires. Six questionnaires were not fully 
answered after checking them, so the total number used as the sample was 117. 

Validity and Reliability Analysis. The validity and reliability of all data must be tested before 
being used in this research. The validity and reliability test was done by using Cronbach's Alpha 
Method. SPSS software version 24 was used to test all of the dependent variables (X1, X2, X3, X4) in 
both parts, which are the User Expectation Part and Application Performance Part, and also the User 
Satisfaction Part as the independent variable (Y). The validity analysis results show that all variables, 
which will be used for further analysis, are valid because the value of "Corrected Item - Total 
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Correlation" is higher than 0.30. The result of the reliability analysis also shows that the instrument 
is reliable and can be used for further analysis because the Alpha Value is higher than 0.70. 

Gap Analysis. The result of the Gap Analysis shows that the performance of the application 
SAP R/3 does not meet the expectation of the user. Figure 4.1 will clearly show the gap between 
User Expectation and Application Performance. 


User Expectaion Variables vs 
Application Performance Variables 


Ss ; 
Variables User Expectation 


—— Application Performance 


Figure 3. Mean Value of User Expectation Variables vs Application 


Paired Sample Test. In this research, we will use the z distribution because the quantity of the 
samples is more than 30. The t values of all the pair variables are less than -1.96. The t value is -5.251, 
which is less than the z value of -1.96 and the significance level is lower than 0.05. Based on the 
analysis, we reject the null hypotheses and conclude that there is a significant gap between the user 
expectation about the effectiveness of the application and the application's performance. 


Table 1. Paired Sample Test Result 


Paired Differences 


Mean Std. Std. 95% Confidence . 
Deviatio Error Interval of the t df Sig. (2- 
n Mean Difference tailed) 


Lower Upper 


PX1 - EX1 -0.314 0.616 0.057 -0.201 -0.427 -5.511 116 0.000 
PX2 - EX2 -0.239 0.576 0.053 -0.134 -0.345 -4.494 116 0.000 
PX3 - EX3 -0.200 0.610 0.056 -0.088 -0.312 -3.548 116 0.001 
PX4 - EX4 -0.504 0.732 0.068 -0.370 -0.638 -7.449 116 0.000 

PX - EX -0.314 0.634 0.059 -0.198 -0.430 -5.251 116 0.000 


Table 2. Analysis of Variance (ANOVA) - First Model 


Model Sum of Squares Df Mean Square F Sig. 
1 Regression 11.471 4 2.868 21.757 | 0.000a 
Residual 14.762 112 0.132 
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Total 26.232 116 
a Predictors: (Constant), Integrated data, Reliable report, Processing time, 


Flow of information between the department 


b Dependent Variable: User Satisfaction 


From the result of the ANOVA test or F test shown in Table 4.2, we can see that the significance 
value is 0.000. This value shows that the regression as a whole is significant. 


Table 3. Regression Coefficient - First Model 


Coefficients 
Unstandardized Standardized 
Model Coefficients Coefficients t Sig. 
B Std. Error Beta 
1 (Constant) 0.341 0.481 0.708 0.480 
Integrated data (x1) 
0.189 0.095 0.144 1.993 0.049 
Reliable report (X2) 
0.239 0.099 0.246 2.399 0.018 
Processing time (X3) 
0.356 0.111 0.332 3.218 0.002 
Flow of information 
between 
departments (X4) 
0.091 0.057 0.141 1.598 0.113 


a Dependent Variable: User Satisfaction 


Table 3 shows that the significance value of variables X1 (integrated data), X2 (reliable report), 
and X3 (processing time) are 0.049, 0.018, and 0.002, which are lower than 0.06 (6%). It means that 
the three variables above are significant. On the other hand, variable X4 (flow of information 
between departments) has a significance value of 0.113, which is higher than 0.06 (6%). It shows that 
variable X4 is not significant. So, variable X4 will not be used in the second analysis. 

Regression Analysis - Second Model. In the Second Model, only three variables are used in 
the analysis: Integrated data, Reliable reports, and Processing time. 


Table 4. Analysis of Variance (ANOVA) - Second Model 


Model Sum of Squares Df Mean Square F Sig. 

1 Regression 11.134 3 3.711 27.776 0.000a 
Residual 15.098 113 0.134 
Total 26.232 116 


a Predictors: (Constant), Integrated data, Reliable report, Processing time 
b Dependent Variable: User Satisfaction 


The ANOVA or F test result shown in Table 4.4 shows that the significance value is 0.000. This 
value shows that the regression as a whole is significant. 
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Table 5. Regression Model Summary - Second Model 


Model Summary 
Model R R Square Adjusted R Std. Error Change Statistics 
Square of ; the R Square FChange df Df2 Sig. F 
Estimate Change 1 Change 
1 0.651a 0.424 0.409 0.366 0.424 27.776 3 113 0.000 


a Predictors: (Constant), Integrated data, Reliable report, Processing time 
b. Dependent Variable: User Statisfiction 


The value of R Square from Table 4.5 is 0.424. This means that 42.4% Y (user satisfaction) can 


be explained by X1 (integrated data), X2 (reliable report), and X3 (processing time). Meanwhile, the 
rest (100% - 42.4% = 57.6%) explains by other reasons. 


Table 6. Regression Coefficient - Second Model 


Coefficients 
Unstandardized Standardized 
Model Coefficients Coefficients T Sig. 
B Std. Error Beta 
1 (Constant) 0.317 0.484 0.654 0.514 
Integrated 
data (X1) 0.183 0.095 0.140 1.918 0.058 
Reliable 
report (X2) 0.277 0.097 0.285 2.849 0.005 
Processing 
time (X3) 0.412 0.106 0.384 3.895 0.000 


a Dependent Variable: User Satisfaction 


Table 6 shows that the three variables used in the second model have significance values lower 
than 0.06 (6%), X1= 0.058, X2 = 0.005, and X3 = 0.000. Thus all three variables are significant. 


First, Data Integrity is essential to support the system so it can give all user data. Second, Real- 
Time means that if data is changed in one department, other users in another department get the 
same data simultaneously. So, the data process is done at the same the changes occur. 


CONCLUSION 


The conclusions that can be drawn based on the gap analysis result, and multiple regression 
results, are : 

1. Based on the first part of the questionnaire, respondents have enough experience using PCs, their 
educations, and their experiences working in PT. ABC, the result of the research, can be 
responsible. 

2. There is a gap between user expectations and application performance. The gap between user 
expectation and application performance is in all aspects analyzed in this research: integrated 
data, reliable reports, processing time, and flow of information between departments. This can 
be interpreted as an indication that the user is dissatisfied with the application and is not 
practical. 
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Based on the multiple regression analysis results, three variables significantly affect user 
satisfaction in using SAP R/3: integrated data, reliable reports, and processing time. Moreover, 
only one variable, the flow of information between departments, is not significantly affecting 
user satisfaction with using SAP R/3. It means that the users have high expectations about the 
three variables but are not concerned about the last variable (flow of information between 
departments) 


Recommendations. There are some recommendations, which are : 


1. User involvement is essential to be considered in the application development process. 

2. Input and opinions from the user can become valuable input in the improvement of the 
application. 

3. The application development team should focus on the three variables that significantly affect 
user satisfaction: integrated data, reliable reports, and processing time. 

4. New users need to be trained before getting a user id and using SAP R/3 so that the 
application can give optimum benefit for the user or the company. 

5. Scheduled training is needed to improve user capability in using SAP R/3 and application 
development. So, users can use SAP R/3 to give 100% support on their job. 

6. Further research needs to be done after some development of R/3. 

7. Further research needs to be done to identify other factors that might be affected user 
satisfaction with using SAP R/3. 
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